Machine learning and statistical approaches to classification – a case study

The advent of information technology has led to the proliferation of data in disparate databases. Organisations have become data rich but knowledge poor. Users need efficient analysis tools to help them understand their data, predict future trends and relationships and generalise to new situations in order to make proactive knowledge-driven decisions in a competitive business world. Thus, there is an urgent need for techniques and tools that intelligently and automatically transform these data into useful information and knowledge for effective decision making. Data mining is considered to be the most appropriate technology for addressing this need. Datamining is the process of extracting or “mining” knowledge from large amounts of data. Regression analysis and classification are two datamining tasks used to predict future trends. In this study, we investigate the behaviour of a statistical model and three machine learning models (artificial neural network, decision tree and support vector machine) on a large electricity dataset. We evaluate their predictive abilities based on this dataset. Results show that machine learning models, for this real world dataset, outperform statistical regression while artificial neural network outperforms support vector machine and decision tree in the classification task. In terms of comprehensibility, decision tree is the best choice. Although not definitive this research indicates that certainly these machine learning methods are an alternative to regression with certain datasets

Interval type-2 Atanassov-intuitionistic fuzzy logic for uncertainty modelling

This thesis investigates a new paradigm for uncertainty modelling by employing a new class of type-2 fuzzy logic system that utilises fuzzy sets with membership and non-membership functions that are intervals. Fuzzy logic systems, employing type-1 fuzzy sets, that mark a shift from computing with numbers towards computing with words have made remarkable impacts in the field of artificial intelligence. Fuzzy logic systems of type-2, a generalisation of type-1 fuzzy logic systems that utilise type-2 fuzzy sets, have created tremendous advances in uncertainty modelling. The key feature of the type-2 fuzzy logic systems, with particular reference to interval type-2 fuzzy logic systems, is that the membership functions of interval type-2 fuzzy sets are themselves fuzzy. These give interval type-2 fuzzy logic systems an advantage over their type-1 counterparts which have precise membership functions. Whilst the interval type-2 fuzzy logic systems are effective in modelling uncertainty, they are not able to adequately handle an indeterminate/neutral characteristic of a set, because interval type-2 fuzzy sets are only specified by membership functions with an implicit assertion that the non-membership functions are complements of the membership functions (lower or upper). In a real life scenario, it is not necessarily the case that the non-membership function of a set is complementary to the membership function. There may be some degree of hesitation arising from ignorance or a complete lack of interest concerning a particular phenomenon. Atanassov intuitionistic fuzzy set, another generalisation of the classical fuzzy set, captures this thought process by simultaneously defining a fuzzy set with membership and non-membership functions such that the sum of both membership and non-membership functions is less than or equal to 1. In this thesis, the advantages of both worlds (interval type-2 fuzzy set and Atanassov intuitionistic fuzzy set) are explored and a new and enhanced class of interval type-2 fuzzy set namely, interval type-2 Atanassov intuitionistic fuzzy set, that enables hesitation, is introduced. The corresponding fuzzy logic system namely, interval type-2 Atanassov intuitionistic fuzzy logic system is rigorously and systematically formulated. In order to assess this thesis investigates a new paradigm for uncertainty modelling by employing a new class of type-2 fuzzy logic system that utilises fuzzy sets with membership and non-membership functions that are intervals. Fuzzy logic systems, employing type-1 fuzzy sets, that mark shift from computing with numbers towards computing with words have made remarkable impacts in the field of artificial intelligence. Fuzzy logic systems of type-2, a generalisation of type-1 fuzzy logic systems that utilise type-2 fuzzy sets, have created tremendous advances in uncertainty modelling. The key feature of the type-2 fuzzy logic systems, with particular reference to interval type-2 fuzzy logic systems, is that the membership functions of interval type-2 fuzzy sets are themselves fuzzy. These give interval type-2 fuzzy logic systems an advantage over their type-1 counterparts which have precise membership functions. Whilst the interval type-2 fuzzy logic systems are effective in modelling uncertainty, they are not able to adequately handle an indeterminate/neutral characteristic of a set, because interval type-2 fuzzy sets are only specified by membership functions with an implicit assertion that the non-membership functions are complements of the membership functions (lower or upper). In a real life scenario, it is not necessarily the case that the non-membership function of a set is complementary to the membership function. There may be some degree of hesitation arising from ignorance or a complete lack of interest concerning a particular phenomenon. Atanassov intuitionistic fuzzy set, another generalisation of the classical fuzzy set, captures this thought process by simultaneously defining a fuzzy set with membership and non-membership functions such that the sum of both membership and non-membership functions is less than or equal to 1. In this thesis, the advantages of both worlds (interval type-2 fuzzy set and Atanassov intuitionistic fuzzy set) are explored and a new and enhanced class of interval type-2 fuzz set namely, interval type-2 Atanassov intuitionistic fuzzy set, that enables hesitation, is introduced. The corresponding fuzzy logic system namely, interval type-2 Atanassov intuitionistic fuzzy logic system is rigorously and systematically formulated. In order to assess the viability and efficacy of the developed framework, the possibilities of the optimisation of the parameters of this class of fuzzy systems are rigorously examined. First, the parameters of the developed model are optimised using one of the most popular fuzzy logic optimisation algorithms such as gradient descent (first-order derivative) algorithm and evaluated on publicly available benchmark datasets from diverse domains and characteristics. It is shown that the new interval type-2 Atanassov intuitionistic fuzzy logic system is able to handle uncertainty well through the minimisation of the error of the system compared with other approaches on the same problem instances and performance criteria. Secondly, the parameters of the proposed framework are optimised using a decoupledextended Kalman filter (second-order derivative) algorithm in order to address the shortcomings of the first-order gradient descent method. It is shown statistically that the performance of this new framework with fuzzy membership and non-membership functions is significantly better than the classical interval type-2 fuzzy logic systems which have only the fuzzy membership functions, and its type-1 counterpart which are specified by single membership and non-membership functions. The model is also assessed using a hybrid learning of decoupled extended Kalman filter and gradient descent methods. The proposed framework with hybrid learning algorithm is evaluated by comparing it with existing approaches reported in the literature on the same problem instances and performance metrics. The simulation results have demonstrated the potential benefits of using the proposed framework in uncertainty modelling. In the overall, the fusion of these two concepts (interval type-2 fuzzy logic system and Atanassov intuitionistic fuzzy logic system) provides a synergistic capability in dealing with imprecise and vague information

Interval type-2 intuitionistic fuzzy logic system for time series and identification problems - a comparative study

This paper proposes a sliding mode control-based learning of interval type-2 intuitionistic fuzzy logic system for time series and identification problems. Until now, derivative-based algorithms such as gradient descent back propagation, extended Kalman filter, decoupled extended Kalman filter and hybrid method of decoupled extended Kalman filter and gradient descent methods have been utilized for the optimization of the parameters of interval type-2 intuitionistic fuzzy logic systems. The proposed model is based on a Takagi-Sugeno-Kang inference system. The evaluations of the model are conducted using both real world and artificially generated datasets. Analysis of results reveals that the proposed interval type-2 intuitionistic fuzzy logic system trained with sliding mode control learning algorithm (derivative-free) do outperforms some existing models in terms of the test root mean squared error while competing favourable with other models in the literature. Moreover, the proposed model may stand as a good choice for real time applications where running time is paramount compared to the derivative-based models

Interval type-2 Atanassov-intuitionistic fuzzy logic for uncertainty modelling

This thesis investigates a new paradigm for uncertainty modelling by employing a new class of type-2 fuzzy logic system that utilises fuzzy sets with membership and non-membership functions that are intervals. Fuzzy logic systems, employing type-1 fuzzy sets, that mark a shift from computing with numbers towards computing with words have made remarkable impacts in the field of artificial intelligence. Fuzzy logic systems of type-2, a generalisation of type-1 fuzzy logic systems that utilise type-2 fuzzy sets, have created tremendous advances in uncertainty modelling. The key feature of the type-2 fuzzy logic systems, with particular reference to interval type-2 fuzzy logic systems, is that the membership functions of interval type-2 fuzzy sets are themselves fuzzy. These give interval type-2 fuzzy logic systems an advantage over their type-1 counterparts which have precise membership functions. Whilst the interval type-2 fuzzy logic systems are effective in modelling uncertainty, they are not able to adequately handle an indeterminate/neutral characteristic of a set, because interval type-2 fuzzy sets are only specified by membership functions with an implicit assertion that the non-membership functions are complements of the membership functions (lower or upper). In a real life scenario, it is not necessarily the case that the non-membership function of a set is complementary to the membership function. There may be some degree of hesitation arising from ignorance or a complete lack of interest concerning a particular phenomenon. Atanassov intuitionistic fuzzy set, another generalisation of the classical fuzzy set, captures this thought process by simultaneously defining a fuzzy set with membership and non-membership functions such that the sum of both membership and non-membership functions is less than or equal to 1. In this thesis, the advantages of both worlds (interval type-2 fuzzy set and Atanassov intuitionistic fuzzy set) are explored and a new and enhanced class of interval type-2 fuzzy set namely, interval type-2 Atanassov intuitionistic fuzzy set, that enables hesitation, is introduced. The corresponding fuzzy logic system namely, interval type-2 Atanassov intuitionistic fuzzy logic system is rigorously and systematically formulated. In order to assess this thesis investigates a new paradigm for uncertainty modelling by employing a new class of type-2 fuzzy logic system that utilises fuzzy sets with membership and non-membership functions that are intervals. Fuzzy logic systems, employing type-1 fuzzy sets, that mark shift from computing with numbers towards computing with words have made remarkable impacts in the field of artificial intelligence. Fuzzy logic systems of type-2, a generalisation of type-1 fuzzy logic systems that utilise type-2 fuzzy sets, have created tremendous advances in uncertainty modelling. The key feature of the type-2 fuzzy logic systems, with particular reference to interval type-2 fuzzy logic systems, is that the membership functions of interval type-2 fuzzy sets are themselves fuzzy. These give interval type-2 fuzzy logic systems an advantage over their type-1 counterparts which have precise membership functions. Whilst the interval type-2 fuzzy logic systems are effective in modelling uncertainty, they are not able to adequately handle an indeterminate/neutral characteristic of a set, because interval type-2 fuzzy sets are only specified by membership functions with an implicit assertion that the non-membership functions are complements of the membership functions (lower or upper). In a real life scenario, it is not necessarily the case that the non-membership function of a set is complementary to the membership function. There may be some degree of hesitation arising from ignorance or a complete lack of interest concerning a particular phenomenon. Atanassov intuitionistic fuzzy set, another generalisation of the classical fuzzy set, captures this thought process by simultaneously defining a fuzzy set with membership and non-membership functions such that the sum of both membership and non-membership functions is less than or equal to 1. In this thesis, the advantages of both worlds (interval type-2 fuzzy set and Atanassov intuitionistic fuzzy set) are explored and a new and enhanced class of interval type-2 fuzz set namely, interval type-2 Atanassov intuitionistic fuzzy set, that enables hesitation, is introduced. The corresponding fuzzy logic system namely, interval type-2 Atanassov intuitionistic fuzzy logic system is rigorously and systematically formulated. In order to assess the viability and efficacy of the developed framework, the possibilities of the optimisation of the parameters of this class of fuzzy systems are rigorously examined. First, the parameters of the developed model are optimised using one of the most popular fuzzy logic optimisation algorithms such as gradient descent (first-order derivative) algorithm and evaluated on publicly available benchmark datasets from diverse domains and characteristics. It is shown that the new interval type-2 Atanassov intuitionistic fuzzy logic system is able to handle uncertainty well through the minimisation of the error of the system compared with other approaches on the same problem instances and performance criteria. Secondly, the parameters of the proposed framework are optimised using a decoupledextended Kalman filter (second-order derivative) algorithm in order to address the shortcomings of the first-order gradient descent method. It is shown statistically that the performance of this new framework with fuzzy membership and non-membership functions is significantly better than the classical interval type-2 fuzzy logic systems which have only the fuzzy membership functions, and its type-1 counterpart which are specified by single membership and non-membership functions. The model is also assessed using a hybrid learning of decoupled extended Kalman filter and gradient descent methods. The proposed framework with hybrid learning algorithm is evaluated by comparing it with existing approaches reported in the literature on the same problem instances and performance metrics. The simulation results have demonstrated the potential benefits of using the proposed framework in uncertainty modelling. In the overall, the fusion of these two concepts (interval type-2 fuzzy logic system and Atanassov intuitionistic fuzzy logic system) provides a synergistic capability in dealing with imprecise and vague information

Extended Kalman filter-based learning of interval type-2 intuitionistic fuzzy logic system

Fuzzy logic systems have been extensively applied for solving many real world application problems because they are found to be universal approximators and many methods, particularly, gradient descent (GD) methods have been widely adopted for the optimization of fuzzy membership functions. Despite its popularity, GD still suffers some drawbacks in terms of its slow learning and convergence. In this study, the use of decoupled extended Kalman filter (DEKF) to optimize the parameters of an interval type-2 intuitionistic fuzzy logic system of Tagagi-Sugeno-Kang (IT2IFLS-TSK) fuzzy inference is proposed and results compared with IT2IFLS gradient descent learning. The resulting systems are evaluated on a real world dataset from Australia’s electricity market. The IT2IFLS-DEKF is also compared with its type-1 variant and interval type-2 fuzzy logic system (IT2FLS). Analysis of results reveal performance superiority of IT2IFLS trained with DEKF (IT2IFLS-DEKF) over IT2IFLS trained with gradient descent (IT2IFLS-GD). The proposed IT2IFLS-DEKF also outperforms its type-1 variant and IT2FLS on the same learning platform

Time series forecasting with interval type-2 intuitionistic fuzzy logic systems

Conventional fuzzy time series approaches make use of type-1 or type-2 fuzzy models. Type-1 models with one index (membership grade) cannot fully handle the level of uncertainty inherent in many real world applications. The type-2 models with upper and lower membership functions do handle uncertainties in many applications better than its type-1 counterparts. This study proposes the use of interval type-2 intuitionistic fuzzy logic system of Takagi-Sugeno-Kang (IT2IFLS-TSK) fuzzy inference that utilises more parameters than type-2 fuzzy models in time series forecasting. The IT2IFLS utilises more indexes namely upper and lower non-membership functions. These additional parameters of IT2IFLS serve to refine the fuzzy relationships obtained from type-2 fuzzy models and ultimately improve the forecasting performance. Evaluation is made on the proposed system using three real world benchmark time series problems namely: Santa Fe, tree ring and Canadian lynx datasets. The empirical analyses show improvements of prediction of IT2IFLS over other approaches on these datasets

Interval type-2 intuitionistic fuzzy logic system for non-linear system prediction

This paper presents an approach to prediction based on a new interval type-2 intuitionistic fuzzy logic system (IT2IFLS) of Takagi-Sugeno-Kang (TSK) fuzzy inference. The gradient descent algorithm (GDA) is used to adapt the parame- ters of the IT2IFLS. The empirical comparison is made on the designed system using two synthetic datasets. Analysis of our results reveal that the presence of additional degrees of freedom in terms of non-membership functions and hesitation indexes in IT2IFLS tend to reduce the root mean square error (RMSE) of the system compared to a type-1 fuzzy logic approach and some interval type-2 fuzzy systems

Hybrid learning for interval type-2 intuitionistic fuzzy logic systems as applied to identification and prediction problems

This paper presents a novel application of a hybrid learning approach to the optimisation of membership and non-membership functions of a newly developed interval type-2 intuitionistic fuzzy logic system (IT2 IFLS) of a Takagi-Sugeno-Kang (TSK) fuzzy inference system with neural network learning capability. The hybrid algorithms consisting of decou- pled extended Kalman filter (DEKF) and gradient descent (GD) are used to tune the parameters of the IT2 IFLS for the first time. The DEKF is used to tune the consequent parameters in the forward pass while the GD method is used to tune the antecedents parts during the backward pass of the hybrid learning. The hybrid algorithm is described and evaluated, prediction and identification results together with the runtime are compared with similar existing studies in the literature. Performance comparison is made between the proposed hybrid learning model of IT2 IFLS, a TSK-type-1 intuitionistic fuzzy logic system (IFLS-TSK) and a TSK-type interval type-2 fuzzy logic system (IT2 FLS-TSK) on two instances of the datasets under investigation. The empirical comparison is made on the designed systems using three artificially generated datasets and three real world datasets. Analysis of results reveal that IT2 IFLS outperforms its type-1 variants, IT2 FLS and most of the existing models in the literature. Moreover, the minimal run time of the proposed hybrid learning model for IT2 IFLS also puts this model forward as a good candidate for application in real time systems

Interval type-2 A-intuitionistic fuzzy logic for regression problems

This paper presents an approach to prediction based on a new interval type-2 Atanassov-intuitionistic fuzzy logic system (IT2AIFLS) of Takagi-Sugeno-Kang (TSK) fuzzy inference with neural network learning capability. The gradient descent (GD) algorithm is used to adapt the parameters of the IT2AIFLS. The empirical comparison is made on the designed system using some benchmark regression problems - both artificial and real world datasets. Analyses of our results reveal that IT2AIFLS outperforms its type-1 variant, other type-1 fuzzy logic approaches and some type-2 fuzzy systems in the regression tasks. The reason for the improved performance of the proposed framework of IT2AIFLS is because of the introduction of non-membership functions and intuitionistic fuzzy indices into the classical IT2FLS model. This increases the level of fuzziness in the proposed IT2AIFLS framework, thus providing more accurate approximations than AIFLS, classical type-1 and interval type-2 fuzzy logic systems

Comparing Performance Potentials of Classical and Intuitionistic Fuzzy Systems in Terms of Sculpting the State Space

This paper provides new application-independent perspectives about the performance potential of an intuitionistic (I-) fuzzy system over a (classical) TSK fuzzy system. It does this by extending sculpting the state space works from a TSK fuzzy system to an I-fuzzy system. It demonstrates that, for piecewise-linear membership functions (trapezoids and triangles), an I-fuzzy system always has significantly more first-order rule partitions of the state space-the coarse sculpting of the state space-than does a TSK fuzzy system, and that some I-fuzzy systems also have more second-order rule partitions of the state space-the fine sculpting of the state space-than does a TSK fuzzy system. It is the author's conjecture that, for piecewise-linear membership functions (trapezoids and triangles): It is the always-significantly greater coarse (and possibly fine) sculpting of the state space that provides an I-fuzzy system with the potential to outperform a TSK fuzzy system; and, that a type-1 I-fuzzy system has the potential to outperform an interval type-2 fuzzy system. Index Terms-intuitionistic fuzzy sets, intuitionistic fuzzy systems, TSK fuzzy systems, rule partitions, sculpting the state space